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(54) PICTURE INFORMATION INPUT DEVICE 

(57)Abstract: 

PROBLEM TO BE SOLVED: To provide a picture 
information input device capable of obtaining depth 
information flexibly corresponding to processing for 
converting a subject into an objective picture format 
SOLUTION: Depth information extracting processing 
executes separating processing and an image picked up 
at a main point 1110 indicating the image pickup 
reference point of a camera head becomes a background 
separated picture 1100 in which a background 1102 and 
a subject 1101 are separated by the separating 
processing. The existence probability of a main subject 
in voxel space 1 1 20 is voted by respective segments 
1 130 to 1 134. In the voting action, the inside part of the 
segments 1 130 to 1 140 is regarded as the subject and 
'1' is integrated, and when the inside part of the 
segments 1130 to 1140 exists in the voxel space, '1' is 
added. Thus voting, e.g. integrating processing, by plural 
segments started from the outline of the subject and 
passing the main point is executed. 
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ESSf < tf 9 , Wtft 1 1 30~l 1 40iD 

[ 0 0 7 6 ] =fe^Sr^x.^ei5 t, x JiigKLfcJ: 5 

n-rfcfc*>«3r*aaasfTfeix5 0 ^jfeojgfu-m, * 



[0 0 7 7] #J;tl4. 0 6 (a) ;# * 7 

~y Ki*s^Sr-rJtefc>*>m*Sr«aiwffiai 2 0 0-1 
2 0 3IC^XL^*5t,J|^-f 5 - tlcioT-fe-^-y-ffil 2 
4 0- 1 2 4 3 \Z.nt>tlTz<&&?S-»mz : t(DlifcWfrb& 
±& 1200—1203 feltiJ-^t^SW 1 2 1 0 IC 
[OltfXfaXfib&Wlfcl 2 2 0 — 1 2 2 7#!§|7!p*K ^(D 

jo &mwBi 2 2 o~i 2 2 7 ©rtfflya^fcg^flct-rs 

SSflcJ;orSS#ll^t Lt«l 2 3 0iis#f)ii 

[0 0 7 8] C<7?#^^bWi&^#:£ffl^5.r il;:,fc 
oT\ 2 1 0 -hl-(i^^rc>ttai^mi: 

LTabcdefgh T?lS*tv5®^ 1 2 3 0 /jqfibfr 

[0 0 7 9] C<D#^tVfc^^l 2 3 0f4, EI6 (b) 
Ui^-rj: 5I£, 2 3 l Sr^tfJg^lc<c 

20 bixz>Km\s.m<v®i^mm\zfi:<w< ^tws. ■ 
[0080] jiifi£DiK?q-ct±, *aasr*p< Kiro 

- ro*aa* 3&5Eifttc:fT 5*§-a-lci4, t»Jx<iB17lc 
^■f-J;5l-x ^■fcA^WfflyrofiEg (X, Y, Z) d><b 

5. zro^fm^-fe^KroiS^ (o, 0, 0) a>?> 

30 (X0, Y0, Z0) *-T?ift!?igL.fT;b;h,5. 

[0 0 8 1 ] C^T?, i£15£Jx5£;£ Ox, y) ir-T5 
x, y(4&© (4) , (5) , (6) 5W>e>5fc#t> 

[0 0 8 2] 
[14] 



fx' "1 fi 0 0 "] f co» e 

l^Z' J lj) sin<D coa cd J^-sinf> 



0 1 id e 

1 0 

0 co* e „ 



cos £ -sin $ 
s ic ^ co 1 <P 
0 0 



a 
b 
c 



[0 0 8 3] 
[S5] 

x = f * x ' y 2. ' 

[0 0 8 4 ] 
[&6] 

v = f * y y z • 

fc*:U (a, b, c) ttifili ioroM«4r^i-„ 
[0 0 8 5] -trojKKffiB (x, y) ^iragR^tdHR:? 



[0 0 8 6] ^K)J;5t-5J<^-fe/^raffl!)*»e>lII^I-|S]ft 

50 t u-camsixSo 
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[0 0 8 7 ] fits. _hi£ro#j-e«u #SDB*^f>aji5^ 
©isiitii, rro&ii&ai^g^Lfc&at urge 

[0 0 8 8 ] RttJg^<Dffi¥«c|c*f1-5, EI 6 

(b) izmT&mkmmcD&w&M^^^xmi text* 

Ss LTV* 5. 

[0089] ironeff^coK^ffic^-rsis^flg 

li. 1217 Ic^-TJ: 51c, iS^J&^aUciotf-S^-fe/i^ 
Mro— SJ5 2 l o Jii^SISr^-rmJt/^HI-.fco-c 2 

[0 0 9 0 ] Coa^M*«r 3^7cfi*)l-^1-t, E18IC 

tut, mm^nm^cim^tix^^o *mx-\z. *^ 

*§H3&tf* 8 7 If ft if fc ft t) . 

[0091] ^^ud-B^A^roUtT^ltffltttii 
jaai^o^TEi 9ftv>ua 1 1 Sr^fsuftasibiftiwi" 

^^-TEI, Ell 0f±,^.7°u— h-7^^>^Sr^i-[gl, El 

^i-EJT'fo5„ 
[0 0 9 2 ] if, *xU*Hfca»ibHfT#1t$ft&-tttti 

-r s&a^mic-o^-cEi 9 l/.£ a* "bta^-rs,, 

[0093] El 9 £#!$1-2> lc N BHfe^^e-y d&jjrt^i-v 

TV 5, RiitfgU 1 OR U}IT\ Wfel 1 ORiV^) 
t*iLt^fl^«tSrMt5Lltl 1 0L (£Jt 

T, BH£l 1 0 L tv?) (±, xyVttffiflil l l L. 

mmMMltimi 17L> w-^-B«*K£:^tSmB151 1 
2lc#x.bH5„ P^lc, B&l 1 0RI1 ^-yv^titi 

SSI 1 1 R> 17R, x-Tua-w&Mfc 

£f*itiSBi l 2tc4-A^ix5o 
[ 0 0 9 4 ] =&^s<^t&!ijlHS 1 1 1 L, 111RI1, *f 
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^tAtUasi 1 7 L, 117RI1 WE&f$o ! HHgt£l^7£-f 
s^aa^tf p„ *x u-^-B^M/S-Stttaa 1 nii, # 

[O 0 9 5]#iy^tt[tia!lllL, lllRtiS 

zMzmm^v^m&itifciSMi&ni 1 3ic4-x.ib 

*u ^■■v'Sm&LltifoM&SaUi l 3 f±#ilMferoii3&co*i- 

10 [0 0 9 6] ^yifW&ttf&Mtti&ni 1 3T-ttttj£ix 
£;h.fcttJSI!l«iifcKI&*&SS&l 1 4U:-5-x.t>*v 

5„ ^*«ia$i5i i 4it, ^vvmtottfc&iftomui 1 
3 -etttb £ ntzttfcMm t . * 7- \s*mQm&M.mm 

l l 2-cttSB$ti.fc*t^BI«i:rom(c:?Sasfe5d^ : S : ^ 
^ffii^rSrBfcirl-Sit&a&fT 5. rro^a-C(±, x y ^ 

wg-m&tttum 1 3T-tsm$ti.7t*f^Be#i:. 

^7hB&ttf££**tH&B l l 2f«)lii$McMlSiff.tii 
20 -ffcbftvtt, ^<7>m&B«liff*Itt#tevi: LT# 

[0097] m^&mn 114 ro«taigmi±, 

*2;3gfSl 1 6 IC-^-^btvSo 
[0 0 9 8] ^fc&Wfci&mm 1 1 5I±, 

[0 0 9 9] ^rtf #tf m^niii^mU 1 1 6 (4, 

tsi@#.a^ = « saawjsar-^ff # tt#5>* ^SLtti l , 
f>Ji5. cco=^a!lSt-ovNTii (1) s;r!JiW 

[0100] #^.^t*tUgB 1 1 7 L, 117 R-CP5£ 

^^fcifseBw^m^iiWiE^-^^tasaagBi 1 sic 

«JE^-r?^tH*aaei51 l 8liIi/<7 7'- 

f^W&lrzb b tii^s ^-vr d 8 8 0 -efii(±j$tvfc^ 
S^b^ifrotf $8*>ib*7< K 1 »^§&, ^SilS#> 

50 tT#tf®^*stffi*&aeBi 1 6i-^^<bHs 0 
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[0 10 1] =^-y VWmMfcfi.miiiMl 1 3lc*5 

^t5*fI£^.tIf^^±i*■itlcoV^Tl211 0 L^/i* ht&W 

-r 5„ 

[0102] ^^K&ftJ&^tliJtmi 1 3 IC&ItS*}' 
roft3t#j ft trot LTI4, fyyu-F-? 

M-l H-l 

R U b) =£ X I l R(a b) (i 
i=0 j=fl 



*© (7) Kt?a$tv5®«) -t-esia^n, mmn 

(a, b) (.KT© (8) 5£-C#J6ib;ft<3) «5«/Mc<£ 



[0103] 
[&7] 

[0104] 
[18] 



R A = (M-N+ 1 ) 



■ i) 



<i,j) I 



[0105] ^^^tti^&t LTH, tf'J^f* bx<- h 
[0106] B^— h 7-f /u^&ffi^S A^iPi^^ 



5K f (i, j) , juiiH^g (i, j) t-rst, au^s 

«!g (i, j) t4&w O) sesfctt (io) ssa^sfc 

[0107] 



B<iv j) - scirt[{f U,j>— f U*l. +{f f li.5+1)) 2 ] 



[0108] 



abs{£(i.j) - 



[tl 0] 

-£ ,3*1) )«obs(£ (i-t-1, 3 ) — £(i, jtl) ) 



v— ^7 -f -^©©-a-, (id, (i2), 
(i 3) atj^ibjRfc^a-ua. 

[0109] 

imii] 



[0110] 
[tl 2] 



- 1 


0 


1 




0 


2 


— 1 


0 


1 


— 1 


— 2 





fy = 0 0 0 

1 2 1 

[0111] 
[S13] 

[0112] W±ro^fetc,t^tf s Eli n^-ti:? 



[01 13] &m&-C(D?.7- u^BtetJ: 0#?>Ja 

30 /cllfTMtiarofliSI-ov^Ell 2 £#RBLfca s ?>lftW 

-rs. mi 2i4#«^^.7 x i-3t-a^^ibcD^tT^tfasr 
m-s-r s «t a © m n £ * -ria $> s . 

[0 114] ^^-c^ftS^fc^^u^Bifero-t^ 
tv^t>l4s ±&©l^^©;*7 i i^B&;4>ibro|iffi'tt 

[0 115] =S-ta^i75^^u^-B^d»ib#ibixfc«:ff# 

tt^i 2 oi±, eii 2\z7jk-r± ^fts-rs*^ 7~ 
40 ? h'<D®.w.jjfa\nm 1 2 5 1 1 tic, m&&&&^& 1 

2 l 

[0116] l 2 1 14, €-m^Slc»e> 

# tf « Sr«E©gc- $ tufcffi«l^«>1S*l-^!ft-r S - t 
50 lc3E|fti-5*-ffit UTI4, 77^ ^IftifeifSrfflV^-C^- 
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[0 117] ffi— mMJk<oWfi^^ Sfttt, 

*u, mntffimm&^&i 2 214, 

tfttf?ai 2 0iifilt5^?rftft5i £ tld, -tro^k 
©JS^R5c7>*P B TSrfTV\ £<bld, b t 3 o^XJrO 

[0118] fttfttf $a«#^© l 2 2 T*j££-£-£*tfcl& 

ibro^-i?^— -^3 >ff$&i: t &lda?^£f5 12 4 ir-^-^ib 
*V, 2 4(4, SE-a-^nfc^fTtttai:^-^^- 

[0119] &id, ttt fa^-a-g^ 1 2 2 o&gid 

oi^TEIl 3 ^HSLft/S^fftf^-f 5„ Ell 3 I4E1 1 2 

[0120] =g-tl,^tc*j(t5*fTtttiai: Lt, 113 
fciL, -toioiiA C20000) r-^i-HtT^tf ?8 

-efo9, -tO&tTttf $ftld(4 2 0 0 2 0 ~ 2 0 0 2 2 4 
■C©tf^Li5^-4^xv>5 0 iiKO 10(40 (2 0 00 1) 
t*^i-SltTtttm-c*&f3, ^t>l:i©loliD (2 0 0 
0 2) T^mtTttf^T-foSo fc*5, ##!JT-(4, ffi!* 
IbOfcfeld, Htf #tf$&(4r - 0 <D 2^j£M^T-^$ 
*t, ^¥ft:(4*tt*-^IC^b^7 , £V^t(Di:1-2) s , 
[0121] m&s 3tS^A^ro^tT#tf^ A (2 0 
000) , O (20001) , □ (20002) h 

ivxisy^ ^(Dffimt> s s&mi$.2 o o 1 oid*;^, fg 

[0122] Li^U &tT£ft£&2 00 2 0- 2 0 0 2 
2 (4ffiC0ti/S^b#'b^citf $ftldifc Ct^fc 

o»S[^i4 s flill-ltL-t3St±iL-CV^fc'9i-2)o 

frit-g-m, #jE:;&ttffl&gfi-?:©3£v*»,&it$atdj^s 

T, -hi6 Lfci 5 (C *tT#tffa2 0020—2002 

[0123] HtTtiW$a2 0 0 2 0-2 0 0 2 2 (O'ffitS 
ttttteV>i:aittS*l-x CW^tTttt?ft2 0 0 2 0~ 2 0 
0 2 2(4fdirom^:w^fi : tita^lfflLT^$^5„ 
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<b, Cl<7)5£ff #tf#2 0 0 2 0 — 2 0 0 2 2 14ft!l<75^©f 

^UT-(4ef4*-[a]ld*5it2)ffi,w»Bff tv^ ^^efe-^ia*^- 

[0124] ;octj>b, t t 3ta^.^t>ro|l 

tTttf m-WM-tzw®. 2 0 0 1 0 "C 2 o^JbdifB] 
tft^i-Jt^-SrSSfflU 1 ofc*lt±# <S*5*tT# 
tfm(4^S$tv5^i(r^5o *^!)T-(4, Htfttf^2 
0 0 2 0~-2 0 0 2 2i^S^h5. ^ WE&^OggSId 

;o temw-ftm^hh,, cnmm.irm^.tcmn^m &&&& 
[oi2 5] mmv£(r>&v^n$ ffi m.z 0020 — 20 

0 2 2^^*$^fc^ld(4, H?rfT#t#$&, A (2 0 0 0 
0) , O (20001) , □ (20002) fah¥-i$>$: 

tw^tfeieittt* (20003) -e^-r 

[01 26] ft*T#timj4fi:jgfK}£ 2 0 0 1 0 ft-?#<b 

« * m v > r -t <omw<v wn t tt a 5 tftfl^Ksidfr 
[0127] &id, jptD^aa, 1 o<D®^«iaoftoid^ 

BfW^-fH^Pfl-i-S^gl^fcD, d^ro^^^ ! gildo^^-rllI 

1 4*3£vtmi 5 sr#Rgu'jte* s e>ittWi"5o mi4i4# 

Wtd^-t-S, El sii + ra^MM^^^^i-Bi-rfos. 
#Ift^T*J4, sftB^toffi^-fbrofcfeld, 3^^t>©tf® 

[0128] #tg,*C^x i/jMj^d^roSltTttfffiroiBE 

30 -e-*aaid*jv^-c, 121 14 (a) ici^i-i^ic, lorots 

M. (t) -t-#<b^fcHtT #tf«(4r t (0. 0) -Cfc 

5 , &wta^ ( t + 6 t ) -e#t>tufci5rtTtrW» r t + 

5t (8, 0) (4, Ell 4 (b) Id^-f-J; 5 Id, gt— $ 
Hfc*f6)i^a5t« r't + 5t (9, «) i<t5. 
[0 12 9] ^kv^-c^ Ell 4 (c) Id^-Tipid, 

^ttar t (0, *) h't + st (0, <*>) t^^e 

ix^ix^BffWld ( i 0, j 0) -ftv-CV^S i. El l 4 

(d) Id^-f-i 5ld, Tix ( i 0, j 0) #0#itildJ; 
o-C*tT#tt#r t (0, <f>) lr't + «t (0, 
40 <t>) ir(4(5(Sfiia-§-tD$tvfc^Sld^6 0 

[0130] * * ^mm&m&ftiw m 1 td^i-) 

tiftiftS^foS t#, ^fiffWTft (i 0, j 0) (4/h$ 

[0131] ±sero-f n&4i^CD (14) ^ldS^5#^[ 

[0132] 
Mfcl 4] 
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17 18 

H-l K-l 

Q(i. d) = S 2 | r' (iij) (k.l) - r' t**t (ktl) , 

k=0 1=0 



- <Dit<£©Q(OP^ *^J->03QS:#x.5 ( i 0, j 0) ^ 
iH#tb£iK -to (i 0, j 0) Sr-f^gt LX&frZ 

mmmmzh,. m^timi 4 u) i^-t-j: 5 i-i&tf 

ttWffir t (9 , <f>) ir't+5t (9, <*> ) ir^lS 

[0133] crofife-a-fci-tt^fffeti-fc^l^, ID— 10 
ffftthli, ^O|^-^cog^^tft>tv5 title, * 

[0134] r. <Dm-g.<nvm-T?n, m-M.tnm^ti 
oypmn, (is) sc^fcw (1 6) 5t-e^-f-gi# 

[0135] 
[S15] 

[0136] 
[*1 6] 

B < ' o- ' i> 2 + *>< e 0 - + ci^,,— < e2 

fcfcU El, e 2liS2pffi, a, b, c, dliillSI- 
iIiR£^fc#J$C-Cfo5 0 b = c = l. a = 2t 

1-5^ilcJ;oX, S§B6<D>8v H^St LX i: 9 WU&\z¥i]'fiE 

[0 13 7] <^lCl*i6b^fc^C0MS/i^ffiWA 5 ff*5^ 30 

[0 13 8] HfTiMtfar t (9, *) l±. Ell 5 IC^ 
1-J: 5 I-, OEPT, ( i 0, j 0) V7 h Ztltz r ' t 
+ 5 t (9, «*) tt©Pp-C*ive*l.£$*U &fT#ti?« 
r t (9, <f> ) t r ' t + 6 t (9, « ) tt±, 1 &t& 

IS r t (9, * ) tr't + 5t (9, 0) i dati"* 

iV Cro^W-r — ft HfT ttfffi r n e w btt 40 

So 

[0 13 9] ^rotW*feiLTI4, WiWUm. ^7*7 
[0 14 0] CLOJ:?^ ^BRMfiK5>^ibOS^WSr 

Xt± , if{g> i/v/rol ^CSiiS ttfft Id g-5 # If ^tt^tev > 



[0141] i&mfrbto&ftzm&kxT^m 
m*b<omft$^mk<Dffi-&\z:^>\,^xm 1 6 £r#f8Lfc 
frbwm-rz>o m\ 6i*m%fr bco&nz tauter u 

[0 14 2] Sfl5^e>oSttTf m^t* : ri'*m®.fch 

'D&ftzffimtvffi'fri&mxte. eh 8 ic^-f-i 
(6SB*>t>ro5ifTtit*ttai*aa# i (mi 6 (a) ic^ 

#tfSattJtii*&S# 2 (Si 6 (b) X%bhtL 

CWifa^;b*l::J:oT-t©SrcE(::ra:, 1316 (c) Id 
^i-J;?lc. &g# l -C^fjttfcMfftfljffi (^yf-i- 

[0 14 3] l©Si 6 (c) i>f>wy>iJ: 51-, U 

fT^tfiai 0 0 0 1 , 1 0 0 0 2 [±|Sa5j»J; t> *MB!]ldfiZ: 

lt5i t* 5 ^*^, Mr#tf?&i 0 0 0 1 , 1 0 0 0 

2 aqsj ib fr<o&mz. £ t> ntSBigs j: n n-m i-fefi-t s t n 

TtblXZfrb, ^tf^tWai 0 0 0 1 , 1 0 0 0 2tti 

ffls^v^ mie*tu utTtit^i o o o 3ii$aa$ 

fflStu, SfcUtTtttai 0 0 4 liSU^StuSo ^co- 
ill, $&?l5^?>©ft?T^t*«l-*SV-'-C, -^lddh05l4iS 

5ttftS<5< tro-Cfo-So 

[0 14 4] LMot «|q5«SO*fT#tf«J: !J*MHI 

frb b<r>m ^sc^aicfflv^s^t) ott^cosffl^f 
^Sr-irtfiK^a^tTto^ tro^a*g*, ei i 6 

(d) ie^1-J:?l^ S^lce.^-cmib^^litTttW 

[0145] (3iifero^2Mffi) ^ic. &mw<Dmm<r> 
m2mm^^^xm.mi-^, 
[0146] *mMv>wm\-i. MM<vmn&Mkmcm 

^A&aSrtT^. 

[0 14 7] AffWl-lis ^iSrc>mi?i?S-x?»±^ IK^fr 
<D\HUti*hM.Xi£?-t^ft\zi%— I- i ^g:D5Sii«ta 
^tT^^, i«^a»ibroenfT&as!e:>-f*£--rfc5i&s 

(±^<, *^ISrofl5ffiT?l±x ^tJ^-li*^ K l l^jft 

V>ffl!)OB!T-e(±lPT-fo l 9, Kl-ifitt< I5t*2^ 3 
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[0148] croa^iti-.tsssi^aiciioT, & 

m*mm<o% i ^® t [si c x. p i-ff -s. 

ioi4 9] mfcfomzT&m *38w©siiifero 
ms^fgico^xta^-rs,, 

[0150] *mm<r>mm\±. mmco^immtmcm 

SSIfea&fT?,, 

[0151] ^teiftlchSu Hfficom lMmr-li. 

[0 15 2] Jz^W^teSrffiV^i tlcjzot, JLii^ 
[0153] 

f 5. 

[0 15 4] W*3S2BB«roB^tf#A^^Slc: t t^ 

[El l ] *!8WroSi«1f aA^iSSOHJSwm l ff?ftgw 
[0 2] El l ropHfttt mArtgSrofMBfcfl}ja£^1-:7' 



(11) #I?9- 2 3 1 3 6 9 

.20 

[12 3] EliroSi«ffi«A^^firol¥iW*«}fig^-t-y 

[1214] E) 2 ^i>.= ^ hn-^Wfif/Sfc^-r^n 

[12)5] S^^iS'-fe^fW&fflVN/c^tfttt^ttffi^ 

[12)8] H 7£DSM«Lai-J:5#ISI^t5lSr*-ria-Cfo 
[1219] tiaJM^ft^bllfTttS^SrttttJi-S^aigSSr 

^-tia-efos,, 

[010] h^y^-v^^r^l-ElffoS,, 

[El 1 1 ] *=r u-^-Hft*^ ©HfT tff$aw3?{tjSrSEIil& 

Wic^-tig-c&s,, 

[12112] #-m^^^^-^-jii^*^ro*tf#ff#SriK-a- 
20 1-5*aa©Bftix?r^i-|2l-C'fo5 0 

[013] Ell 2(D*ff#tf$a©C-g-^l£l 2 2iCiStf 5 

[eh 4] £&&*T^3rm@.t>-b<Dmft$ftm<om-& 

[Ell 5] f W^ffitW^&^^-TEl-CfoS. 

[eii 6] mmt^hnmrr^mmt^T-u^-m^Lt^tD 

[^^rotKW] 
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